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LONG-TERM  GOALS 

The  long-term  technical  goal  is  to  design,  develop  and  test  the  next  generation,  primitive  equation 
ocean  model  for  high-resolution  scientific  (ROMS:  Regional  Ocean  Modeling  System)  and  operational 
(TOMS:  Terrain-following  Ocean  Modeling  System)  applications.  This  project  will  improve  the  ocean 
modeling  capabilities  of  the  U.S.  Navy  for  relocatable,  coastal,  coupled  atmosphere-ocean  forecasting 
applications.  It  will  also  benefit  the  ocean  modeling  community  at  large  by  providing  the  current 
state-of-the-art  knowledge  in  physics,  numerical  schemes,  and  computational  technology. 

OBJECTIVES 

The  main  objective  is  to  produce  a  tested  expert  ocean-modeling  framework  for  scientific  and 
operational  applications  over  a  wide  range  of  spatial  (coastal  to  basin)  and  temporal  (days  to  seasons) 
scales.  The  primary  focus  is  to  implement  the  most  robust  set  of  options  and  algorithms  for  relocatable 
coastal  forecasting  systems  nested  within  basin-scale  operational  models  for  the  Navy.  The  system  is 
unique  in  that  it  is  the  only  community  framework  that  includes  the  adjoint -based  analysis  and 
prediction  tools  that  are  available  in  Numerical  Weather  Prediction  (NWP),  such  as  4-dimensional 
variational  data  assimilation  (4D-Var),  ensemble  prediction,  observation  sensitivity  and  impact, 
adaptive  sampling,  and  circulation  stability  and  sensitivity  analysis.  ROMS  is  freely  distributed 
(http://www.myroms.org)  to  the  Earth’s  modeling  community  and  has  thousands  of  users  worldwide. 

APPROACH 

The  structure  of  TOMS  is  based  on  ROMS  because  of  its  accurate  and  efficient  numerical  algorithms, 
tangent  linear  and  adjoint  models,  variational  data  assimilation,  modular  coding  and  explicit  parallel 
structure  conformal  to  modern  computer  architectures  (both  cache-coherent  shared-memory  and 
distributed  cluster  technologies).  Currently,  both  ROMS  and  TOMS  are  identical  and  continue 
improving  and  evolving.  ROMS  remains  as  the  scientific  community  model  while  TOMS  becomes  the 
operational  community  model. 

ROMS/TOMS  is  a  three-dimensional,  free-surface,  terrain-following  ocean  model  that  solves  the 
Reynolds-averaged  Navier-Stokes  equations  using  the  hydrostatic  vertical  momentum  balance  and 
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Boussinesq  approximation  (Haidvogel  et  al.  2000,  2008;  Shchepetkin  and  McWilliams,  2005,  2009). 
The  governing  dynamical  equations  are  discretized  on  a  vertical  coordinate  that  depends  on  the  local 
water  depth.  The  horizontal  coordinates  are  orthogonal  and  curvilinear  allowing  Cartesian,  spherical, 
and  polar  spatial  discretization  on  an  Arakawa  C-grid.  Its  dynamical  kernel  includes  accurate  and 
efficient  algorithms  for  time-stepping,  advection,  pressure  gradient  (Shchepetkin  and  McWilliams 
2003,  2005),  several  subgridscale  parameterizations  (Durski  et  al.,  2004;  Warner  et  ah,  2005)  to 
represent  small-scale  turbulent  processes  at  the  dissipation  level,  and  various  bottom  boundary  layer 
formulations  to  determine  the  stress  exerted  on  the  flow  by  the  bottom. 

Several  adjoint-based  algorithms  exist  to  explore  the  factors  that  limit  the  predictability  of  the 
circulation  in  regional  applications  for  a  variety  of  dynamical  regimes  (Moore  et  al.,  2004,  2009). 
These  algorithms  use  the  ideas  of  Generalized  Stability  Theory  (GST)  in  order  to  identify  the  most 
unstable  directions  of  state-space  in  which  errors  and  uncertainties  are  likely  to  grow.  The  resulting 
singular  vectors  can  be  used  to  construct  ensembles  of  forecasts  by  perturbing  initial  and  boundary 
conditions  (optimal  perturbations)  and/or  surface  forcing  (stochastic  optimals).  Perturbing  the  system 
along  the  most  unstable  directions  to  the  state-space  yields  information  about  the  first  (ensemble 
mean)  and  second  (ensemble  spread)  moments  of  the  probability  density  function.  Given  an 
appropriate  forecast  skill  measure,  the  circulation  is  predictable  if  low  spread  and  unpredictable  if 
large  spread. 

ROMS/TOMS  uniquely  supports  three  different  4D-Var  data  assimilation  methodologies  (Moore  et 
al.,  2011a,  b):  a  primal  form  of  the  incremental  strong  constraint  4D-Var  (I4D-Var),  a  strong/weak 
constraint  dual  form  of  4D-Var  based  on  the  Physical-space  Statistical  Analysis  System  (4D-PSAS), 
and  a  strong/weak  constraint  dual  form  of  4D-Var  based  on  the  indirect  representer  method  (R4D- 
Var).  In  the  dual  formulations,  the  search  for  the  best  ocean  circulation  estimate  is  in  the  subspace 
spanned  only  by  the  observations,  as  opposed  to  the  full  space  spanned  by  the  model  as  in  the  primal 
formulation.  Although  the  primal  and  dual  formulations  yield  identical  estimates  of  the  ocean 
circulation  for  the  same  a  priori  assumptions,  there  are  practical  advantages  and  disadvantages  to  both 
approaches  (Moore  et  al.,  2011a,  b,  c).  To  our  knowledge,  ROMS/TOMS  is  the  only  open-source, 
ocean  community-modeling  framework  supporting  all  these  variational  data  assimilation  methods  and 
other  sophisticated  adjoint-based  algorithms. 

There  are  several  biogeochemical  models  available  in  ROMS.  In  order  of  increasing  ecological 
complexity  these  include  three  NPZD-type  models  (Franks  et  al.,  1986;  Powell  et  al.,  2006;  Fiechter  et 
al.,  2009),  a  nitrogen-based  ecosystem  model  (Fennel  et  al.,  2006,  2008),  a  Nemuro-type  lower  level 
ecosystem  model  (Kishi  et  al.,  2007),  and  a  bio-optical  model  (Bissett  et  ah,  1999). 

ROMS  includes  a  sediment-transport  model  with  an  unlimited  number  of  user-defined  cohesive  (mud) 
and  non-cohesive  (sand)  sediment  classes  (Warner  et  al.,  2008).  Each  class  has  attributes  of  grain 
diameter,  density,  settling  velocity,  critical  stress  threshold  for  erosion,  and  erodibility  constant.  A 
multi-level  bed  framework  tracks  the  distribution  of  every  size  class  in  each  layer  and  stores  bulk 
properties  including  layer  thickness,  porosity,  and  mass,  allowing  the  computation  of  bed  morphology 
and  stratigraphy.  Also  tracked  are  bed-surface  properties  like  active-layer  thickness,  ripple  geometry, 
and  bed  roughness.  Bedload  transport  is  calculated  for  mobile  sediment  classes  in  the  top  layer. 

ROMS  is  a  very  modem  and  modular  code  written  on  F90/F95.  It  uses  C-preprocessing  to  activate  the 
various  physical  and  numerical  options.  The  parallel  framework  is  coarse-grained  with  both  shared- 
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memory  (OpenMP)  and  distributed-memory  (MPI)  paradigms  coexisting  in  the  same  code.  Because  of 
its  construction,  the  parallelization  of  the  adjoint  is  only  available  for  MPI.  Several  coding  standards 
have  been  established  to  facilitate  model  readability,  maintenance,  and  portability.  All  the  state  model 
variables  are  dynamically  allocated  and  passed  as  arguments  to  the  computational  routines  via 
dereferenced  pointer  structures.  All  private  arrays  are  automatic;  their  size  is  determined  when  the 
procedure  is  entered.  This  code  structure  facilitates  computations  over  nested  grids. 

WORK  COMPLETED 

The  major  overhaul  of  ROMS  to  include  nesting  capabilities  was  completed.  Due  to  its  complexity, 
the  nesting  developing  was  divided  in  three  sequential  phases.  Phase  I,  released  as  ROMS  3.5  on  the 
25th  April  2011,  included  substantial  modifications  of  the  numerical  kernels  (NLM,  TLM,  RPM,  and 
ADM)  to  allow  a  generic  treatment  of  the  spatial  horizontal  operators  in  the  nesting  contact  regions. 
Phase  II,  released  as  ROMS  3.6  on  the  23rd  September  2011,  included  an  overhaul  of  the  lateral 
boundary  conditions  to  facilitate,  in  a  generic  way,  their  processing  or  not  in  applications  with  nested 
grids.  Phase  III,  released  as  ROMS  3.7  on  the  18th  April  2013,  included  the  data  managing  and  time¬ 
stepping  infrastructure  for  one  or  more  nesting  layers. 

Currently,  three  types  of  nesting  capabilities  are  supported  in  ROMS:  (i)  refinement  grids  which 
provide  increased  resolution  (3:1,  5:1,  or  7:1)  in  a  specific  region;  (ii)  mosaics  which  connect  several 
grids  along  their  edges,  and  (iii)  composite  grids  which  allow  overlap  regions  of  aligned  and  non- 
aligned  grids  (Warner  et  ah,  2010).  The  mosaic  and  composite  grid  code  infrastructures  are  identical. 
The  differences  are  geometrical  and  primary  based  on  the  alignment  between  adjacent  grids.  All  the 
mosaic  grids  are  exactly  aligned  with  the  adjacent  grid.  In  general,  the  mosaic  grids  are  special  case  of 
the  composite  grids.  The  nesting  algorithms  are  flexible  enough  to  allow  complex  nested  grid 
configurations  in  coastal  applications  by  permitting  computations  on  various  nested  grid  classes 
(refinement,  mosaic,  and  composite)  and  nesting  layers  (refinement  and  composite  grid  combinations). 

The  4D-Var  data  assimilation  algorithms  were  updated  to  include:  spatial  convolutions  on  geopotential 
surfaces  when  modeling  spatial  error  covariances,  time  error  covariance  modeling,  a  background 
quality  control  of  the  observations  (Anderson  and  Jarvinen,  1999),  expected  analysis  and  forecast 
errors  (Moore  et  ah,  2012),  diagnostic  information  content  (Desroziers  et  ah,  2009),  and  a  new 
Lanczos-based  minimization  algorithm  (RBCG:  Restricted  B-preconditioned  Conjugate  Gradient)  for 
the  dual  formulation  (Gurol  et  ah,  2013).  The  mathematical  iterates  and  convergence  rates  of  the 
RBCG  algorithm  are  the  same  as  the  primal  fonnulation  Gauss-Newton  algorithms.  That  is,  the  weak 
constraint  (dual  fonnulation:  4D-PSAS,  R4D-Var)  minimization  is  as  affordable  as  the  strong 
constraint  (primal  fonnulation:  I4D-Var)  minimization.  This  makes  it  practical  to  use  4D-PSAS  and 
R4D-Var  in  operational  applications. 

Several  new  algorithms  were  added  to  the  General  Stability  Analysis  (GSA)  toolkit  to  compute  4D-Var 
Hessian  eigenvector  analyses  on  the  initial  conditions  and  model  forcing. 

We  held  a  very  successful  workshop  at  the  Windsor  Atlantica  Hotel,  Rio  de  Janeiro,  Brazil,  October 
22-25,  2012.  As  in  the  past,  several  tutorials  were  offered  on  basic  and  advanced  ROMS  algorithms.  In 
addition,  we  had  a  special  session  on  modern  observational  and  modeling  systems  with  several  invited 
speakers.  We  offered  informal  4D-Var  data  assimilation  workshop  in  Santa  Cruz,  CA,  July  18-30, 
2013.  We  have  several  attendees  from  US  and  Brazil  applying  4D-Var  to  their  own  application. 
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RESULTS 


An  example  of  one-way  nested-grids  is  shown  in  Fig.  1  for  the  Gulf  of  Mexico.  Two  grids  are 
considered:  a  coarse  grid  for  the  full  Gulf  of  Mexico  with  an  average  horizontal  resolution  of  15km 
(Fig.  la)  and  a  fine  grid  for  the  Northern  Gulf  of  Mexico  at  3km  (Fig.  lb).  The  coarse  to  fine  grids 
have  a  refinement  ratio  of  1 :5.  The  coarse  grid  initial  conditions  and  lateral  boundary  conditions  are 
derived  from  the  North  West  Atlantic  ROMS  solution.  The  fine  grid  is  initialized  from  the  coarse  grid. 
Both  grids  are  forced  with  atmospheric  fields  derived  from  the  European  Centre  For  Medium-Range 
Weather  Forecasts  (ECMWF)  ERA-Interim,  3-hour  dataset.  River  runoff  is  included  along  the 
Alabama,  Mississippi  and  Louisiana  coasts. 
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Figure  1:  A  ROMS  nested  grid  application  in  the  Gulf  of  Mexico  showing  the  simulated  sea  surface 
high  (m)  seven  days  after  initialization.  The  average  horizontal  resolutions  are:  (a)  15km  for  the 
coarser  grid  and  (b)  3km  for  the  fine  grid.  The  nesting  grid  refinement  ratio  is  1:5.  Notice  the 
better-resolved  features  in  the  fine  Northern  Gulf  of  Mexico  grid. 
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Figure  1  shows  an  instantaneous  sea-surface  height  for  both  coarse  and  fine  grids  seven  days  after 
initialization.  A  well-developed  Loop  Current  and  a  soon  to  be  detached  eddy  can  be  seen.  Notice 
that  this  eddy  is  centered  on  the  southern  lateral  boundary  of  the  fine  grid. 
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Figure  2:  Sea  surface  high  (m)  and  vertically  integrated  currents  (m/s)  for  a  one-way  nesting 
application  in  the  Gulf  of  Mexico  seven  days  after  initialization.  The  fine  (white  outline)  is  a  1:5 
refinement  ratio  from  coarse  grid  (red  outline).  The  fine  grid  solution  is  overlaid  on  top  of  coarse 
grid.  The  fine  grid  current  vectors  have  been  sampled  every  5  grid  points  for  clarity. 

Although  this  is  a  one-way  nesting  simulation,  the  flow  of  information  between  coarse  and  fine  grids 
occurs  at  every  time-step  and  the  lateral  boundaries  are  well  behaved.  Figure  2  shows  the  same  fields 
but  with  the  fine  grid  (b)  on  top  of  the  coarse  grid  (a).  In  addition,  the  figure  includes  the  vertically 
integrated  currents  with  the  fine  grid  vectors  sampled  at  every  fifth  grid  point  for  consistency  and 
clarity.  Notice  that  sea-surface  height  contours  are  continuous  between  both  grids  making  the 
transition  almost  invisible  without  small-scale  contamination.  This  example  shows  the  advantages  of 
using  nesting  in  regional  grids  with  very  active  dynamical  regimes. 
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IMPACT/APPLICATIONS 


This  project  will  provide  the  ocean  modeling  community  with  a  freely  accessible,  well  documented, 
open-source,  terrain-following,  ocean  model  for  regional  nowcasting  and  forecasting  that  includes 
advanced  data  assimilation,  ensemble  prediction,  and  analysis  tools  for  adaptive  sampling  and 
circulation  dynamics,  stability,  and  sensitivity. 

TRANSITIONS 

The  full  transition  of  ROMS/TOMS  to  the  operational  community  is  likely  to  occur  in  the  future. 
However,  the  ROMS/TOMS  algorithms  are  now  available  to  the  developers  and  scientific  and 
operational  communities  through  the  website  http://www.myroms.org/. 

RELATED  PROJECTS 

The  work  reported  here  is  related  to  other  already  funded  ONR  projects  using  ROMS.  In  particular, 
the  PI  (H.  Arango)  closely  collaborates  with  A.  Moore  (adjoint-based  algorithms)  at  University  of 
California,  Santa  Cruz,  A.  Miller  and  B.  Comuelle  (ROMS  adjoint  and  variational  data  assimilation)  at 
Scripps  Institute  of  Oceanography,  and  J.  Wilkin  (Mid-Atlantic  Bight  variational  data  assimilation)  at 
Rutgers  University. 
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